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Abstract 


Remote sensing missions past the era of LANDSAT D require the dissemination 
of high quality image data to users in near real time. Martin Marietta has 
developed a unique approach to onboard processing which is directed at this goal. 
The first step of this approach was the development of an onboard cloud detection 
system which has flown on an aircraft flight test and will fly on the first 
Shuttle experimental pallet. The second step of the approach was the development 
of a Landmark tracker, which has also been flown on an aircraft flight test. 

This paper outlines the results of these two developments and summarizes the 
requirements of an operational guidance and control system capable of providing 
continuous estimation of the sensor boresight position. 

Introduction 


All forecasts of advanced technology and the future space mission models 
have pointed to massive increases in image data return from spaceborne sensor 
platforms designed to provide global monitoring of agriculture, minerals, forest, 
and water resources. Concurrently, the user community is requesting high quality 
image products in a shorter amount of time. Examination of existing and near- 
term mission models reveals that the end to end remote sensing system is ineffic- 
ient. Over 50% and closer to 80% of all data acquired by the Landsat series 
remains unused due to either undesirable effects such as cloud coverage or dis— 
interesting scene content. Also, the turnaround time between data acquisition 
and dissemination to the user can exceed two months due to tremendous processing 
requirements necessary to correct imagery for distortions. This situation is 
intolerable to both NASA and the user community. In summary, two major limita- 
tions of existing remote sensing missions are deterministic acquisition of high 
quality imagery and the timely correction of imagery for distortions. This paper 
outlines an approach to remote sensing which will meet future mission goals by 
overcoming these limitiations . The approach is centered around two subsystems. 
The first subsystem provides real time classification of features within a scene 
so that onboard decisions affecting data acquisition can be made. The second 
subsystem incorporates a landmark tracker into a state of the art navigation 
system in order to continuously predict the sensor boresight position in earth 
fixed coordinates. 


t James W. Lowrie is a senior engineer at Martin Marietta Aerospace, Denver 
Division, working in the Advanced Automation Technology area. 

ft John E. Myers is a Professor of Electronics Engineering Technology at Metro- 
politan State College, Denver, Colorado, and consultant to Martin Marietta 
Aerospace, Denver Division. 
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Deterministic Data Acquisition 


In order to solve the problem of acquiring only desirable scenes, it is 
necessary to define the features which are desirable or undesirable and then to 
develop a system which will automatically classify scenes according to their 
content. For remote sensing missions, it is certainly necessary to distinguish 
clouds from other features, but it is also desirable to separate others such as 
vegetation, bare earth, and water. For example, a mission dedicated to water 
pollution monitoring has no desire to acquire bare earth or vegetation scenes. 
Therefore, for this application it is necessary to discriminate between water 
and other classes. Table I presents a list of mission models and the types of 
data selection criteria they might use. 


Table I. Data Selection Criteria for Advanced Mission Models 


Mission Model 


Data Selection Criteria 


Biomass Estimation 
Flood Detection 
Forest Fire Detection 

Water Pollution Monitoring 

Ice Mapping 

General Remote Sensing 
(Landsat) 


Cloud 

Vegetation 

Cloud 

Water 

Cloud 

Vegetation 

Fire 

Cloud 

Water 

Water/Land Interface 

Cloud 

Snow 

Ice 

Cloud 


The Feature Identification and Location Experiment (FILE) was first con- 
ceived in 1976 as the first segment of a truly autonomous remote sensing system 
(Ref. 1). The experiment, which has flown on an aircraft flight test in early 
1980 and is scheduled to fly on shuttle OFT-2, is designed around the concept 
that generic classes of features may be separated by spectral signature using 
simple algorithms. It is important to note that this experiment eliminates the 
need for detailed ground truth information by avoiding the temptation to separate 
generic clusters into finer detail. The FILE algorithm utilizes the ratio of 
the sensor voltages in two bands centered at .65ym and .85ym. Although the 
observed radiance from a feature is a function of its reflectance, incident 
illumination, and radiance absorption of the medium through which it is viewed, 
the ratio of the radiance at these two wavelengths is reasonably independent of 
all factors except reflectance. This principle is the basis of the FILE system 
and is the key to avoiding the need for ground truth. Figure I shows how various 
feature types can be classified with the algorithm. Water and vegetation can be 
separated on the basis of the ratio alone, However, since the radiance ratio 
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for bare land is essentially the same as for clouds and snow, these features 
must be separated on the bases of absolute radiance. 

Although the FILE experiment has been designed to classify clouds, vegeta- 
tion, bare earth, and water, the technique may be extrapolated to other target 
types as long as the statistics of the signatures are seperable. For example, 
forest fire detection could be implemented using a thermal and visual band. 



Figure I. 99% Confidence Polygons , Sun 41 to 60 Degrees from Zenth 


Image Correction 


The advancement of spaceborne processors has made real time correction of 
imagery a feasible goal for near-term mission models provided the distortions 
can be measured onboard to sufficient accuracy. The primary sources of image 
distortion can be separated into sensor peculiarities, viewing perspective, and 
spacecraft characteristics (Ref. 2). With the development of linear arrays, 
the primary sensor-caused distortions will be the individual placement of 
detector elements and the orientation of the array relative to the sensor prior 
to flight. Viewing perspective, which is a combination of curvature of the 
field of view and look angle geometry , is a slowly varying function 
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of local earth radius and can also be considered deterministic over short 
intervals. The primary error source remaining, therfore, is spacecraft-caused 
distortions. The spacecraft error sources can be catagorized as follows: 

Attitude determination 

Ephemeris prediction 

Misalignment between sensor and body coordinates 

Mathematical inaccuracies in inertial to earth 
fixed coordinate transformation 

These general sources have been broken down in more detail in Table II. 

Table II. Spacecraft Induced Error Sources in Temporal Registration 

Attitude determination 

Star tracker accuracy 

Star tracker configuration 

Knowledge of star tracker misalignment 

Error in star catalogue 

Gyro noise 

Knowledge of gyro bias, nonorthogonality, misalignment 
Numerical accuracy 

Ephemeris prediction 
GPS accuracy 
Numerical accuracy 

Misalignment between sensor and body coordinates 

Knowledge of linear array or scan mirror orientation 
Accuracy of thermal deflection model 
Vibration modes between two coordinates 
Calibration technique and frequency 
Numerical accuracy 

Transformation error between inertial and earth fixed coordinates 
Knowledge of UT1 

Knowledge of earth precession, nutation, polar wander, and 
tidal deformation 
Numerical accuracy 


For the sake of discussion, assume that all the error is due simply to the 
attitude determination system. In order to achieve a temporal registration 
accuracy of 15 meters, it will be necessary to predict the attitude to within 
4 sec as illustrated in Figure II. Accuracy of current state-of-the-art systems 
using the NASA standard star tracker and gyro is 15 sec (2a) as discussed in the 
f, Onboard Attitude Determination System" study (Ref. 3). 
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Figure II. Error Budget for Registration Accuracy of 15m 



Even with the advancement of CCD star trackers, the attitude determination 
capability will be around 6 sec (2a) . Note that two sigma numbers have been 
used here corresponding to 95% of the data. If a one sigma number corresponding 
to 67% of the data is used, the accuracy goal can be met. However, by adding 
just one more error term such as a misalignment between sensor and body coordin- 
ates of 2 sec (2-axis accuracy achievable with an optical alignment cube), the 
total error exceeds the design goal. From the previous discussion, which ignored 
many error sources, it is clear that another approach is required. 

Solution of the temporal registration problem requires that the sensors 
boresight position in earth fixed coordinates be periodically measured. This 
can be accomplished using a correlator which registers known Ground Control 
Points (GCP) within the sensor data. Onboard registration of GCPs allows many 
of the error terms listed in Table II to be accurately estimated in real time. 


Shortly after the experimental definition of FILE, Martin Marietta began the 
development of a landmark tracker or GCP detector centered around our experience 
with terminal guidance systems. The primary function of the landmark tracker is 
to provide periodic measurements of the science sensors boresight position to be 
used as an input to a navigation system. Previous studies (Ref. 4-7) have shown 
that the landmark tracker will not adequately solve for both position and attitude 
without supplemental measurements from another source. For this reason the remote 
sensing navigation system has been configured with a GPS receiver to provide 
position measurements. Another limitation of the landmark tracker operating in 
the visual spectrum is that measurements are sometimes obscured by clouds and no 
measurements can be taken over water. For this reason, two star trackers have 
been added to the configuration to bound the maximum attitude error and to reduce 
the convergence time of the state when GCP sightings are acquired. A block 
diagram of the navigation system is shown in Figure III. 

The registration processor is centered around a Sequential Similarity 
Detection Algorithm (SSDA) first identified by Barnea and Silverman (Ref. 8). 

Other algorithms were considered, but after significant analysis (Ref. 9), re- 
sults indicate that for the advanced Landsat mission model, the SSDA is superior 
to other techniques due to its low probability of false lock, time required for 
registration, and ease of implementation in a hardwired system. 
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- Vehicle State Solution 

- Sensor Boresight 

Position Determination 


**Data Management Processor 

- Data Annotation 

- Image Correction 

- Sensor Pointing 

- Information Extraction 

- Telemetry Management 


Figure III, GPS Detection System Configuration 
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To perform GCP registration it is not necessary to process imagery from 
the entire Field of View (FOV) but only an area whose size ensures the GCP will 
be located within its boundary. 


Let this search area be defined as an LxL area of digital picture elements. 
The image may be defined by a function, S, that describes the gray scale, or 
recorded radiance, in relation to position coordinates, i.e., 

S(i,j) = Wi,j 

where Wj . is the gray scale of the i,j fc ^ picture element of the search area 
» J 

1 £(i, j)£ L. 

Let the ground control point be defined similarly as an MxM area with an image 
function 

G(£,m) = R 1>m 

where Rj , m is the gray scale of the 1 ,m th picture element of the GCP 
1 , m)£ M. 

A sub image (Figure IV) of the SA may be defined as an MxM area whose upper left 
coordinates (n,o) lie in the range 

1 < (n, o)< L - M + 1. 


L 



Figure IV. Svbimage Definition 

A subimage whose upper left coordinates are n,o will be referred to as the njO 1 -* 1 
reference point. 


The sensor data are registered by measuring the similarity between each 
MxM subimage within the search area and the representation of the GCP stored 
onboard. The reference point that produces the highest degree of similarity 
with the GCP is then the best registration of the SA and can be labeled with the 
same earth fixed coordinates as the GCP. 
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The SSDA algorithm may be implemented to detect similarity between a 
reference point and the GCP through the following equation: 


M 1! 

Similarity = £ £ 

i-1 j-1 


(S(i + n, j + 0) - S n>Q ) - ( G (i,j) - G) 


where 

S no = reference point, 

S no = mean value of the subimage located at the njO^ reference point 

G = mean value of the GCP, 

The entire registration process can then be described by the algorithm shown 

in Figure V. 



Figure V. Registration of an Area Landmark 

Approaches to automatic registration have typically been limited by the 
effect of cloud coverage on accuracy and the inability to detect correlator 
false lock. A technique for reducing the effects of cloud coverage was devel- 
oped under an independent research project (Ref. 9). The technique incorporates 
the FILE classification capability into the correlator so that every pixel 
representing a cloud is eliminated from the correlator computation. Results 
indicate that the tolerance for clouds within the search area has increased 
from 10% to 40%. An algorithm was also developed to detect correlation false 
lock. Basically, the algorithm compares the rate of convergence of the correl- 
ation surface with the rate of convergence found when the GCP is correlated 
with itself. If false lock is detected, no registration vector is passed to 
its navigation filter. 
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System Model 


Under contract to NASA-GSFC, Martin Marietta is currently investigating 
the operational requirements of an onboard GCP detection system designed to 
meet the goals of accurate image correction. The analysis is centered around 
a simulation program which models the environment of the spacecraft, generates 
measurements, and estimates the state of the vehicle using an extended Carlson 
square rodt filter. The program was set up to provide analysis of true errors 
rather than simply evaluating the covariance matrix. Although the covariance 
analysis provides a great deal of information, interpretation of results can be 
inaccurate and misleading. For example, there are many cases where the covar- 
iance matrix converges over a period of time while the actual state estimate 
diverges from the true state. A conceptual diagram of the modeling is shown 
in Figure VI. 



Figure VI. Overview of Measurement Models 


The design philosophy behind the measurement models is that the actual 
vehicle state is used with a geometry model to yield an ideal measurement 
vector. This ideal vector is then corrupted with bias, noise, and misalignment 
to provide the actual sensor output. The sensor output is then compensated for 
some estimate of the error terms and is used by the filter to estimate the 
vehicle state. The benefit behind this design approach is that it enables a 
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detailed analysis of sensitivity to misalignments and compensation ability. 

It is also expected that the severe requirements associated with onboard image 
correction will require the onboard estimate of misalignment terms such as 
those between the science sensor and body coordinates. With this approach, it 
will not be difficult to modify the filter to solve for these terms. It is 
possible to understand the mathematics of most of the measurement models simply 
by interpreting Figure VI. However, the landmark tracker model is somewhat 
more complex and is described more fully here. 


The landmark location on the surface of the earth in Local Landmark 
Coordinates (Figure VII) will be a function of the altitude (Ar) above the 
earth’s mean radius.* 


Ml 


0 

0 


However, in earth fixed coordinates, the landmark will have the earth's mean 
radius (rig) added to the altitude. Using the angular transformation from local 
landmark to earth fixed coordinates produces 



r E + a l 


CLCX 

-SL 

-CLSX 


1 

Me = e t l Ml = e t l 

0 

= (r E + A l ) 

SLCX 

CL 

-SLSX 


0 


0 


SX 

0 

CX 


_ 0 _ 


= CF E + A l ) 


CLCX 

SLCX 

SX 


As shown in Figure VII, the position vector of the spacecraft (Ps/C)> when sub- 
tracted from the landmark position in some coordinate frame, will provide the 
measurement vector (M) . 


Mi - (i t e ~ Zs/Ci 

Accounting for hardware misalignments, the same measurement vector in landmark 
tracker coordinates is: 

M& = £ t i ((i t e Mg) - p s/ci) 

= «, t e Me - 2, t i Ms /ci 


From examination of Figure VIII, the unit measurement vector in landmark tracker 
coordinates is: 


U£ 


MJt 

W 



Uix 


CosAVSinAH 

= 

Uiy 


SinAV 


Ulz 


CosAVCosAH_ 


However, the tracker instrument has no sensitivity to projections along its 
boresight axis. Therefore, the tracker response to the unit vector UX will be: 
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lCosAVSinAH 


S/C 



Figure VII 

Landmark Tracker Geometry - 
Sighting Plan 



Landmark Tracker Geometry 

General 
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U£x 


CosAVSinAH 

U£y 


SinAV 

l_ J 




ua = 


producing a H and V as shown in Figure VIII as sensor outputs. Since the 
sensor output will be corrupted by bias and noise, the sensed measurement will be 


Z£ = 

AH S 

= 

AH + bH + v H 


AV S 


AV + by + Vy 


Where • 


bfl, by = Component landmark tracker bias 

V H , V v = Component landmark tracker zero mean random noise, N(0,o 2 ) 
The component biases and standard deviations (a) are user selectables. 


The landmark tracker measurement may be compensated for knowledge of instrument 
bias. The bias knowledge may be a priori or through estimation. The compensated 


A 

Zl = 

1 

> 

o 

1 

— 

AHs 

a 

- bH 

-A 


AVc 

» — 


AV S 

- bv 


Where: 


LM = the landmark being used = f (L , A , Al) 

Al = the altitude of the landmark above the mean radius of the earth 


L = longitude of the landmark 
A = latitude of the landmark 

L = vector position of the landmark relative to the center of the earth 
Zs/C = vector position of the spacecraft relative to the center of the earth 
As/C = altitude of the spacecraft above the mean radius of the earth 
M = measurement vector from the spacecraft to the landmark 
U = unit vector along M 

AH = the landmark tracker horizontal place angular deflection from the 
boresight axis 

AV = the landmark tracker vertical plane angular deflection from the 
boresight axis 


The dynamics model calculates the derivative of the spacecraft navigational 
state, which will be integrated to produce the navigational state vector. This 
is done in part by calculating the total acceleration of the spacecraft due to 
solar pressure and gravitation effects of the sun, moon, and earth, including 
fourth zonal harmonic terms. The total acceleration of the spacecraft can be 
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found by solving the following simultaneous equations: 

Xi = -Xr -p-+ gi(t,x) + ax(t,x) 

X 2 = -X 2 - + g 2 (t,X) + a 2 (t,x) 

X3 = -X 3 * JJ + g 3 (t,X) + a 3 (t,X) 

where 

X = (Xi, x 2 , x 3 ) t 

y = earth gravitational constant (3.98549120E + 14m 3 /sec 2 ) 

R = (X? + X? + Xf)^ 

X \ , X 2 > X 3 = coordinates of spacecraft 

gl, 82 > 83 = accelerations caused by zonal harmonics of earth gravity 
ai , a£> a 3 = solar radiation pressure perturbations, sun and moon gravity 

The position state is advanced in time by numerical integration of the 
equations of motion consisting of external forces acting on the spacecraft. 
Analysis of various integration algorithms has shown that the Runge Kutta Gill 
4th order numerical integration method is optimal for this application. It is 
self-starting, handles variable step sizes, and is sufficiently accurate. The 
Runge Kutta Gill method for numerically integrating differential equations is 
described here: 

The change in the value of the function during the computing interval 
is calculated by 

Ay = j (ki + 2(l-y)k 2 + 2 (l+y)k 3 + k 4 ) 

where 

kx = h*f(t n , y n ) y = / 2/2 

k 2 = h • f (t n + hh, y n + kk ) 

k 3 = h-f(t n + hh, y n + (- J 5 + y)k 1 + (1 - y)k 2 ) 

k 4 = h*f t n + h, y n - yk 2 + (1 +y)k 3 ) 
h = computing interval (seconds) 

t n = time of beginning of computing interval (seconds) 

y n = value of function at beginning of computing interval 

The derivative function f is evaluated four times to calculate the change 
in the function being integrated during the computing interval. 

Software Simulation 

A Ground Control Point Simulation (GCPSIM) program has been configured to 
provide scientific simulations to predict the performance of the GCP detection 
system over a wide range of circumstances. Figure IX is a flow diagram of the 


15-13 


Sequencer 


Establish 
Measurement type 
Measurement time 




Initialization 


Generate 

measurement 

n 

L f . \ 


Integrate filter state, 
state transition matrix, 
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by one step 



Figure IX. Flew Diagram of GCPSIM 
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simulation. GCPSIM has been designed to provide the ability to analyze the 
effect of various measurement sequences. This is especially important when 
studying the effect of GCP spacing, missed GCP sightings, and the expected 
accuracy after traversing a large body of water. The measurement sequencer 
designed for GCPSIM allows any mixture of GCP, GPS, or star tracker measure- 
ments and time delays (periods during which no measurements were made) of any 
length. The sequencer will determine the type of measurement and the time at 
which the measurement should be made. The true vehicle position state is then 
propagated forward to this time by integrating the nonlinear equations of motion 
with some additional process noise to account for modeling errors. The attitude 
state is propagated by looking up the body rates in an attitude profile table 
and integrating these rates. 

The true vehicle state is used along with a measurement model to generate 
an ideal measurement vector. The ideal measurement is then corrupted with 
noise, bias, and misalignment terms and compensated for knowledge of these 
values. This allows a careful analysis of the effect of misalignment on the 
state solution. It is important to understand the effect of bias and misalign- 
ment between the landmark tracker and body axis because this is the largest 
unknown factor contributing to a pointing error. It is possible to calibrate 
the system for these misalignment errors, but it is difficult to model, for 
any length of time, the various processes which cause the misalignment. For 
example, thermal gradients across the vehicle and vibrational modes within the 
flexible structure are complex functions of such things as structural design, 
sun angle, physical properties of the material, and many other factors. These 
processes are the most difficult and least understood of all engineering prob- 
lems. Therefore, significant emphasis will be placed on analyzing their effect 
on pointing accuracy. 

The compensated measurements are used as inputs into an extended square 
root Kalman filter, which estimates the true vehicle state. The extended filter 
propagates the estimated navigation state, the state transition matrix, and the 
process noise array between measurements by integrating the various differential 
equations using a fourth order Runge Kutta Gill process. The estimated attitude 
state is propagated by a gyro model which corrupts the output with gyro drift, 
noise, nonorthogonality, scale factor, and misalignment. The gyro output is 
compensated, in a similar fasion to the measurement model, by subtracting off 
knowledge of these values. 

The estimated state is used to form an estimated measurement which in turn 
is subtracted from the true measurement to obtain a residual. It is this 
measurement residual and a calculated Kalman gain which are used to update the 
state estimate. By comparing the state estimate with the true state, a direct 
error analysis can be performed. The entire process continues until the space- 
craft is propagated forward to the run stop time. 

GCPSIM has been designed to allow maximum flexibility in the analysis of 
an onboard landmark tracker. Types of analyses to be performed under the con- 
tract are indicated in Table III. 
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Table III . Detailed BreakdobM of Analysis 


- Sensitivity to: 


- Accuracy given: 


- Rate of Convergence: 


- Rate of Divergence: 

- Ability to solve for: 


sensor accuracies 
sensor misalignment 
GCP sighting frequencies 
GCP location in FOV 
knowledge of gyro bias, noise, non- 
orthogonality , misalignment 
knowledge of earth fixed coordinates 
measurement sequence 
1/lOth pixel correlation 
1 pixel correlation 
backup system (star tracker) 
after using backup 

using l/10th pixel correlation accuracy 
using 1 pixel correlation accuracy 
when missing GCP sightings 
sensor misalignment 
earth fixed coordinates 


Summary 

Development of a new generation of remote sensing systems has become a 
necessity for both NASA and the user community in order to fulfill the goals of 
future missions. In the past there has been a lack of coordination between the 
scientific user community and the engineers responsible for spacecraft design, 

This has resulted in a physical separation between the design and implementation 
of the science payload and the control system. This design philosophy must 
change if the future mission requirements are to be met. 

The primary emphasis in the guidance and control system must shift from 
simply estimating the ephemeris and attitude of the spacecraft to estimating 
the position of the science sensors FOV on the earth’s surface. This shift of 
emphasis will impact the design of the entire spacecraft. For example, if the 
science sensor is to be used as a primary attitude sensor, it is desirable to 
place the gyro package in close proximity to that sensor in order to reduce the 
misalignment between the two. This suggests that the current Multi-Mission Space- 
craft (MMS) configuration, which provides a physical separation between the pay- 
load and the guidance and control system, will not satisfy the requirements of 
many future remote sensing missions. 

Martin Marietta, under contract to NASA GSFC is developing an approach to 
remote sensing missions which eliminate the separation between the science in- 
strument and the guidance and control system. Preliminary results obtained in 
the analysis of this system show great promise for automation of the end-to-end 
remote sensing process. 
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